Information processing apparatus

ABSTRACT

According to one embodiment, an information processing apparatus includes a decoder which decodes video coded data, and a load information acquisition means which acquires load information required for processing of data other than the video coded data, wherein the decoder predetermines elimination priorities of stepwise filter processing required for decoding the video coded data, obtains a load level from the load information, and in response to the obtained level, eliminates the filter processing in a stepwise manner in accordance with the priorities.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2005-375203, filed Dec. 27, 2005, theentire contents of which are incorporated herein by reference.

BACKGROUND

1. Field

One embodiment of the present invention relates to an informationprocessing apparatus equipped with a function of decoding video codeddata.

2. Description of the Related Art

As a technique standardized for encoding a video, there have beendeveloped: H. 261 and H. 263 of ITU-T (International TelecommunicationUnion, Telecommunication Standardization Division); and MPEG (MovingPicture Experts Group)-1, MPEG-2, MPEG-4 or the like of ISO(International Standardization Organization). As a next generation videoencoding system further developed while inheriting the techniques suchas H. 261 to 263 and MPEG-1 to -4, there is exemplified H. 264 in whichstandardization has been carried out jointly by the ISO and the ITU. Inthis H. 264, as one of an in-loop filters, there is employed ade-blocking filter for mitigating a distortion generated at a blockboundary, and in particular, an image quality improvement effect at alow bit rate is enhanced. It is disclosed by, for example, ITU-TRecommendation H. 264 (2003), “Advanced Video Coding for genericaudiovisual services”, ISO/IEC 14496-10: 2003, “Information technology,Coding of audio-visual objects—Part 10: Advanced video coding” and H.264/AVC textbook (Impress Communications Co., Ltd.)

However, an information processing apparatus equipped with a videodecode processing function that conforms to standardizationspecification based on H. 264 described above, actually, has a high rateof a processing quantity of an in-loop filter, particularly ade-blocking filter in the whole decoding process. Thus, in the casewhere a processing capability of a central processing unit (CPU) or agraphic controller is low or in the case where a whole processing loadis high, decode processing in real time lags behind, and frame missingoccurs or an object motion becomes extremely slow.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

A general architecture that implements the various feature of theinvention will now be described with reference to the drawings. Thedrawings and the associated descriptions are provided to illustrateembodiments of the invention and not to limit the scope of theinvention.

FIG. 1 is a block diagram depicting an example of a basic configurationof one embodiment of an information processing apparatus equipped with afunction of decoding a video according to the present invention;

FIG. 2 is a block diagram depicting a specific example of aconfiguration of a video decoder shown in FIG. 1;

FIGS. 3A and 3B are views each adapted to explain an example of an edgefilter-processed at a de-blocking filter section shown in FIG. 2;

FIGS. 4A and 4B are views each adapted to explain an example of filterprocessing at the de-blocking filter section shown in FIG. 2;

FIG. 5 is a flow chart showing an example of a method for eliminatingprocedures for filter processing of the de-blocking filter section shownin FIG. 2;

FIG. 6 is a flow chart showing a first embodiment of the filterprocessing eliminating method shown in FIG. 5;

FIG. 7 is a flow chart showing a second embodiment of the filterprocessing eliminating method shown in FIG. 6; and

FIG. 8 is a flow chart showing a third embodiment of the filterprocessing eliminating method shown in FIG. 6.

DETAILED DESCRIPTION

Various embodiments according to the invention will be describedhereinafter with reference to the accompanying drawings. In general,according to one embodiment of the invention, an information processingapparatus includes a decoder which decodes video coded data, and a loadinformation acquisition means which acquires load information requiredfor processing of data other than the video coded data, wherein thedecoder predetermines elimination priorities of stepwise filterprocessing required for decoding the video coded data, obtains a loadlevel from the load information, and in response to the obtained level,eliminates the filter processing in a stepwise manner in accordance withthe priorities.

According to an embodiment, FIG. 1 is a block diagram depicting anexample of a basic configuration of one embodiment of an informationprocessing apparatus equipped with a function of decoding a video codeddata according to the present invention. The information processingapparatus shown in FIG. 1 supplies a compressed/encoded video stream(video coded data) to be inputted from a transmission system to a videodecoder 10 using a graphic controller that conforms to the standardspecification based on H. 264, for example; carries out decodeprocessing; and outputs the decoded data. At this time, the videodecoder 10 notifies a processing load required for decoding to a loadinformation acquisition section 11. This load information acquisitionsection 11 acquires information on a rendering processing load and anaudio processing load in addition to a video decode processing load, andthen, notifies the whole load information to the video decoder 10. Thevideo decoder 10 predetermines elimination priorities of filterprocessing required for decode processing, based on parameters requiredfor decode processing; obtains a load level from the load information;and then, eliminates filter processing in a stepwise manner inaccordance with the priorities, in response to the obtained level.

Acquisition of load information can include: a technique of querying aload to an operating system executed on information processing; and atechnique of detecting a load based on a use rate of either a processoror a memory, the processor executing information processing.

FIG. 2 is a block diagram depicting a specific configuration of thevideo decoder 10 described above. In FIG. 2, an input stream is providedas a stream compressed/encoded in accordance with the H. 264 standard,and sent to a variable length converting section 101 (also referred toas an entropy decoding section). This variable length converting section101 carries out variable length decoding of an inputted video encodedstream, and then, generates syntax. A de-quantization section 102 and ade-conversion section 103 generate a residual image from a result ofdecoding the video encoded stream, based on the generated syntax. Acoding mode control section 104 judges a coding mode from the decodingresult of the variable length converting section 101.

In addition, an intra prediction section 105 and an inter predictionsection 106 each generate intra and inter predictive images,respectively, in accordance with a coding mode specified by the codingmode control section 104. The generated predictive image is selectivelysent to a residual adder section 107. This residual adder section 107adds a predictive image from the intra prediction section 105 or interprediction section 106 and a residual image from the de-conversionsection 103, and then, generates a decoded image. The generated decodedimage is referred to in the intra prediction section 105 and sent to ade-blocking filter section 108.

On the other hand, with respect to the de-blocking filter section 108, ade-blocking filter control section 110 presets stepwise processinglevels relevant to the decoded image generated in advance in theresidual adder section 107 in accordance with control information or theinformation relating to quantization parameters inputted from thevariable length decoding section 101 and the inverse quantizationsection 102. Then, this control section sends to the load informationacquisition section 11 the load information required for video decodeprocessing from the information relating to the quantization parameters;receives whole load information from the acquisition section 11; andthen, determines a processing level of the de-blocking filter based onthat load level. The de-blocking filter section 108 eliminates thefilter processing of the inputted decoded image in a stepwise manner,based on that determination. A reconstruction image filter-processed atthe de-blocking filter section 108 is stored in a picture memory 109.The reconstruction image stored in the picture memory 109 is output as adecoded image or is referred to in the inter prediction section 106.

As described previously, the video encoding system that conforms to theH. 264 standard achieves a high compression rate by the combination of anumber of encoding techniques, whereas it has a problem with a largeamount of processing. In particular, as in HD-DVD (high resolution videoimage DVD), in the case of reproducing a video with high resolution,this problem becomes serious. Among the H. 264 decoders, de-blockingfilter processing carries out computation with respect to block edgesevery four pixels, and thus, an amount of processing is very large.Therefore, the present invention is featured by detecting a load of avariety of processing operations carried out in video image processingsuch as DVD reproduction processing; and, in the case where a load ofaudio processing, rendering processing and the like is high, adaptivelyeliminating de-blocking filter processing, thereby reducing an amount ofdecoder processing.

That is, according to the above configured video decoder 10 of thepresent invention, the de-blocking filter control section 110 acquires aparameter “boundary strength” (hereinafter, referred to as “bS”)indicating filter strength as a quantizing parameter, and a clippingvalue “tc” of a pixel change rate; and then, carries out de-blockingfilter processing with respect to a vertical edge (luma, chroma) and ahorizontal edge (luma, chroma) of a 16×16 micro-block as shown in FIGS.3A and 3B, for example. At this time, a boundary (edge), at which achange of a pixel value due to the de-blocking filter processing asshown in FIGS. 4A and 4B is small, is detected from the filter strengthparameter “bS” and the clipping value “tc” of a pixel change rate; andthen, the elimination of the de-blocking filter processing is appliedpreferentially in response to a level of a processing load. In thismanner, it is possible to achieve a reduced number of processing blockswhile preventing an error from being accumulated on a decoded image.

The parameter “bS” of filter strength of the de-blocking filterprocessing relevant to the above 16×16 macro-block is given as follows.

When bS=4, it follows:p′ ₀=(p ₂+2×p ₁+2×p ₀+2×q ₀ +q ₁+4)>>3p′ ₁=(p ₂ +p ₁ +p ₀ +q ₀+2)>>2p′ ₂=(2×p ₃+3×p ₂ +p ₁ +p ₀ +q ₀+4)>>3q′ ₀=(p ₁+2×p ₀+2×q ₀+2×q ₁ +q ₂+4)>>3q′ ₁=(p ₀ +q ₀ +q ₁ +q ₂+2)>>2q′ ₂=(2×q ₃+3×q ₂ +q ₁ +q ₀ +p ₀+4)>>3

When bS=1, 2, or 3, it follows:Δ=clip₃(−t _(c) , t _(c), ((((q ₀ −p ₀)<<2)+(p ₁ −q ₁)+4)>>3))p′ ₀=clip1(p ₀+Δ)p′ ₁ =p ₁+clip3(−tc ₀ , tc ₀, (p ₂+((p ₀ +q ₀+1)>>1)−(p ₁<<1))>>1)q′ ₀=clip1(q ₀−Δ)q′ ₁ =q ₁+clip3(−tc ₀ , tc ₀, (q ₂+((p ₀ +q ₀+1)>>1)−(q ₁<<1))>>1)

Now, with reference to FIG. 5, a description will be given with respectto a method for eliminating a de-blocking filter processing responsiveto a load level according to the present invention.

First, a load state of whole equipment is queried at the time ofstarting decoding of each picture (block S1); and then, it is judgedwhether or not a high load state is established (block S2). In the casewhere the high load state is not established, standard decode processing(including de-blocking filter processing) is carried out (block S3). Inthe case where the high load state is established, decode processinghaving eliminated the de-blocking filter processing in a stepwise manneris executed in response to the corresponding load level, the parameter“bS” indicating filter strength, and the clipping value “tc” of a pixelchange rate (block S4). The above-described processing is repeated untildecoding of all pictures is completed (block S5).

Stepwise elimination of the above filter processing can be achieved inaccordance with first to third embodiments.

First Embodiment

Filter strength parameters bS exist as 0 to 4. The larger valueindicates the higher filter strength. As shown in FIG. 6, afterdetecting a load quantity (block S11), a load level is judged (blockS12). Here, when the load level=1, bS_th=1 is preset; when the loadlevel=2, bS_th=2 is preset; and when the load level=3, bS_th=3 is preset(blocks S131 to S133); and it is judged whether or not the actuallyacquired parameter bS is bS>bS_th (block S14). If bS>bS_th, standarddecode processing (including de-blocking filter processing) is carriedout (block S15). If bS>bS_th is not established, it is judged that ahigh load state is established, and then, filter processing is executedat a load level that corresponds to the bS value (block S16). When theload level is 0, standard decode processing (block S15) is carried out.

Second Embodiment

A clipping value “tc” of a pixel change rate is provided as a parameterthat exists in the case where a filter strength parameter bS is withinthe range of 1 to 3. This parameter indicates a clipping value of apixel change rate with respect to a filter target pixel. Therefore, thelarger clipping value tc indicates the higher filter strength. As shownin FIG. 7, after detecting a load quantity (block S21), a load level isjudged (block S22). Here, when the load level=1, tc_th=1 is preset; whenthe load level=2, tc_th=2 is preset; and when the load level=L, tc_th=Lis preset (blocks S231 to S23L); and it is judged whether or not theactually acquired parameter tc is tc>tc_th (block S24). If tc>tc_th,standard decode processing (including de-blocking filter processing) iscarried out (block S25). If tc>tc_th is not established, it is judgedthat a high load state is established, and then, filter processing isexecuted at a load level that corresponds to the tc value (block S26).When the load level is 0, standard decode processing (block S25) iscarried out.

Third Embodiment

A pixel change rate increases at an edge having higher filter strength.Thus, in the case of eliminating de-blocking filter processing, a pixelerror is accumulated, and then, a difference from a correct decodedimage becomes large. Therefore, de-blocking filter processing relevantto an edge having small filter strength is eliminated preferentially.

An evaluation value “e” obtained by the formula below is a valueproportional to the filter strength “bS”. Therefore, de-blocking filterprocessing is eliminated with respect to an edge having the evaluationvalue “e” that is smaller than a threshold value θ.e=γ×bS+φ×tcθ=f (L) (load level)

In the formula, γ and φ are constants equal to or greater than 0. θ isdetermined by a function f (L) that returns a larger value as a loadlevel increases. In the formula, although “tc” has been used as aparameter, tc0 may also be used.

A flow of processing operation according to the third embodiment isshown in FIG. 8. First, after detecting a load quantity (block S31), aload level L>0 is judged (block S32). When the load level L>0 inadvance, a threshold value θ is defined as θ=f (L) (block S33); anevaluation value “e” is calculated from the above formula (block S34);and the calculated value is compared with the threshold value θ (blockS35). In this comparison, if e>θ, standard decode processing (includingde-blocking filter processing) is carried out (block S36). If e>θ is notestablished, it is judged that a high load state is established, andthen, filter processing that corresponds to that load level is executed(block S37). In the case where the load level L>0 is not established inblock S32, standard decode processing (block S36) is carried out.

As described above, in the first embodiment, assuming that control ofde-blocking filter processing is carried out using a load level and bS,the de-blocking filter processing is executed only at an edge in whichthe higher load level indicates the higher “bS”. In the secondembodiment, assuming that control of de-blocking filter processing iscarried out using a load level and a clipping value “tc” of a pixelchange rate, the de-blocking filter processing is executed only at anedge in which the higher load level indicates the higher “tc”. In thethird embodiment, assuming that control of de-blocking filter processingis carried out using both of a load level and bS and the clipping value“tc” of a pixel change rate, a threshold value θ, which is larger as theload level is higher, and an evaluation value are compared with eachother, and de-blocking filter processing is executed only at an edgehaving an evaluation value “e” that is greater than θ.

In any embodiment, in response to a level of a processing load,de-blocking filter processing can be eliminated preferentially in astepwise manner, and a reduced number of processing blocks can beachieved while preventing error accumulation of a decoded image.

As has been described above, according to the present invention, thepriorities of eliminating filter processing is predetermined based onparameters required for decoding of the video coded data; a load levelis obtained from load information required for processing of data otherthan the video coded data; and in response to that level, filterprocessing is eliminated in a stepwise manner in accordance with thepriorities, whereby a load of filter processing is reduced in a stepwisemanner in response to another data processing load level so as not tohave an effect on a decode processing speed. As a result, whilepromoting image quality improvement at a low bit rate, in-loop filterprocessing is carried out in a stepwise manner, and then, filterprocessing is reduced in response to a load level of a whole system,thereby making it possible to properly carry out decode processing inreal time. In addition, for use in a portable computer, if informationsuch as battery residue or power saving mode is monitored as part ofload information, decode processing can be effectively continued whilereducing power consumption of video decode processing when the batteryresidue is short or when power saving mode is selected.

The decode processing described above is applicable in the case where itis carried out in any of CPU and a graphic controller. In addition, thedecode processing can be achieved as the video decode processingfunction as described above. At the same time, the above decodeprocessing can be achieved as a video decoding method comprising asmeans the characterizing blocks included in the video decoding method.In addition, the above processing can be implemented as a program thatcauses a computer to execute these blocks. In addition, such a programcan be distributed via a recording medium such as CD-ROM or atransmission medium such as the Internet.

While certain embodiments of the inventions have been described, theseembodiments have been presented by way of example only, and are notintended to limit the scope of the inventions. Indeed, the novel methodsand systems described herein may be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods and systems described herein may be made withoutdeparting from the spirit of the inventions. The accompanying claims andtheir equivalents are intended to cover such forms or modifications aswould fall within the scope and spirit of the inventions.

1. An information processing apparatus comprising: a decoder whichdecodes video coded data; and a load information acquisition meansconfigured to acquire load information required for processing of dataother than the video coded data, wherein the decoder predetermineselimination priorities of stepwise filter processing required fordecoding the video coded data, obtains a load level from the loadinformation, and in response to the obtained level, eliminates thefilter processing in a stepwise manner in accordance with thepriorities.
 2. The information processing apparatus according to claim1, wherein the decoder comprises: a prediction means which generates apredictive image in accordance with a coding mode decoded from the videocoded data; an inverse quantization/inverse transformation means whichgenerates a residual image from quantized orthogonal transformcoefficients decoded from the video coded data; a residual adder-whichadds the predictive image and the residual image to generate a decodedimage; and a filter processor which carries out filter processing forreducing a block distortion of the decoded image in a stepwise manner,and the filter processor comprises: detecting a level of a processingload from load information acquired by the load information acquisitionmeans; detecting a boundary at which a change of a pixel value using thefilter processing is small from parameters required for the filterprocessing; and sequentially eliminating boundaries from a boundary atwhich a change of the pixel value is small, in response to a level ofthe processing load, and then, carrying out filter processing.
 3. Theinformation processing apparatus according to claim 2, wherein thefilter processor uses at least one of filter strength and a clippingvalue of a pixel change rate as a parameter required for the filterprocessing.
 4. The information processing apparatus according to claim1, wherein the load information acquisition means queries a load levelto an operating system executed on information processing.
 5. Theinformation processing apparatus according to claim 1, wherein the loadinformation acquisition means detects the load level based on a use rateof at least either of a processor and a memory, the processor executingthe information processing.
 6. The information processing apparatusaccording to claim 1, wherein the decoder inputs and decodes data basedon H. 264 recommended by ITU (International Telecommunication Union,Telecommunication Standardization Division) as the video coded data. 7.An information processing apparatus comprising: a processor whichcarries out decode processing of a video coded data; and a loadinformation acquisition means configured to acquire load informationrequired for processing of data other than the video coded data during aperiod in which the processor executes a program, wherein the processorcomprises: decoding the video coded data to generate a predictive image;de-quantizing and inverse transform a quantized orthogonal transformcoefficient from the video coded data to generate a residual image;adding the predictive image and the residual image to generate a decodedimage; and carrying out filter processing for reducing a blockdistortion of the decoded image in a stepwise manner, and the filterprocessing comprises: detecting a level of a processing load from theload information acquired by the load information acquisition means;detecting a boundary at which a change of a pixel value using the filterprocessing is small, from parameters required for the filter processing;and sequentially eliminating boundaries from a boundary at which achange of the pixel value is small, in response to a level of theprocessing load, and then, carrying out filter processing.
 8. Acomputer-readable memory containing program instructions for decodingvideo coded data, the memory comprising: performing a function ofdecoding the video coded data to generate a predictive image; performinga function of inverse quantization and inverse transforming a orthogonaltransform coefficient quantized from the video coded data to generate aresidual image; performing a function of adding the predictive image andthe residual image to generate a decoded image; and performing afunction of carrying out filter processing for reducing a blockdistortion of each screen of the decoded image in a stepwise manner, thefilter processing comprising: performing a function of detecting a levelof a processing load from load information required for processing ofdata other than the video coded data; performing a function of detectinga boundary at which a change of a pixel value using the filterprocessing is small, from parameters required for the filter processing;and performing a function of sequentially eliminating boundaries from aboundary at which a change of the pixel value is small, in response to alevel of the processing load, and then, carrying out filter processing.